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© CALIBRATION SYSTEM OF VISUAL SENSOR. 

© A system for calibrating a visual sensor in a 
robot system. An arm [*>) of a robot (I) is provided 
with a pattern plate (3. ■« calibration, and calibration 
pattern data (CPDr) <x pattern plate on the baso 
cooro! : naies of the robot are sent from a robot con- 
troller (10) to a visual sense controller (20). The 
visual sensor controller (20) takes the image of th*-.- 
.pattern piat* (3) by a camera (4) and obtains the 
calibration pattern data (CPDc). Calibration data (CD) 
are obtained from these calibiation pattern data 
(CPDr) and (CPDs) to calibiate th<* visual sensor. 
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Technical Field 

The present invention relates to a visual sensor 
calibration method of calibrating a visual sensor in 
a robot system, and more particularly, to a visual 
sensor calibration method of calibrating a plurality 
of cameras or the like. 

Background Art 

In current practice a robot system is provided 
with a visual function to enable it to pick up an 
image of a workpiece by using a camera or the like 
to determine the position of the workpiece, and 
thus carry out an operation such as assembling or 
palletizing or the like. Further, a plurality of cam- 
eras are used to accurately determine the position 
of a workpiece, or to correspond to a plurality of 
robots. 

It is necessary to calibrate the relationship be- 
tween a robot coordinate system and a camera 
coordinate system, to accurately determine the po- 
sition of a workpiece, and when a plurality of cam- 
eras is used, an exclusively designed jig is pre- 
pared and used. 

Such a calibration jig is large, however, the jig 
must be moved by jogging or a similar operation, 
to thereby pick up images thereof by a camera, 
resulting in a poor operability. 

Further, because most calibration jigs are 
large, they incur high manufacturing costs and the 
like, and require a special installation space. 

Furthermore, when a plurality of cameras is 
used, it is difficult to calibrate each camera. 

Disclosure of th Invention 

The present invention is intended to solve the 
above-mentioned probtem, and an object of the 
present invention '«■.• *o provide a calibration method 
for a visual sens /»hich eliminates th^ need for 
moving a jig when implementing a calibration. 

Another object o? the present invention is to 
provide a calibration method for a visual sensor by 
which a visual sensor can be calibrated with a 
simple calibration jig. 

Still another object of the present indention is 
to provide a calibration method for a visual sensor 
which permits an easy calibration of each camera. 

To fulfill the aforementioned objects, the 
present invention provides; 

a visual sensor calibration method of calibrat- 
ing a visual sensor in a robot system, wherein an 
arm of a robot is provided '/"th a pattern plate for a 
calibration, the first calib* .-n pattern data of the 
pattern plate on base coordinates of the robot is 
sent from a robot controller to a visual sensor 
controller, the visual sensor controller obtains the 



second calibration pattern data from the pattern 
plate, and then acquires calibration data from the 
first calibration pattern data and the second calibra- 
tion pattern data, to thereby calibration a visual 

5 sensor. 

A robot controller itself holds accurate position 
data of the pattern plate on the robot coordinates 
system. More specifically, the robot controller 
holds the first calibration pattern data of the pattern 

w plate based on a coordinate position of the end of 
the robot arm and an installation dimension of the 
pattern plate. 

The visual sensor controller reads the first cali- 
bration pattern data from the robot controller via a 

75 communication line, and further, the visual sensor 
controller receives an input of an image of the 
pattern plate from the visual sensor and detects 
each dot of the pattern to thereby obtain the sec- 
ond calibration pattern data. 

20 The calibration data can be acquired by com- 

paring the first calibration pattern data with the 
second calibration pattern data. 

Brief Description of the Drawings 

25 

FIG. 1 is a configuration block diagram of a 
whole robot system for implementing the visual 
sensor calibration method according to the 
present invention; 
30 FIG. 2 is a detailed view of dot patterns on the 
pattern plate; 

FIG. 3 shows an example of two cameras used 
to pick up an image of one pattern plate; and 
FIG. 4 shows an example whereby, in a robot 
35 system comprising four robots and four cam- 
eras, all four cameras are calibrated against a 
robot coordinate system common to all robots. 

Best Mode of Carrying Out of the Invention 

40 

An embodiment of thy present invention is 
described with reference to Ibe drawings. 

FIG. 1 is a configuration block diagram of an 
entiro robot system for implementing the visual 

45 sensor calibration method according to the present 
invention In FIG I, in order to give a brief descrip- 
tion of the flow of the calibration pattern data, only 
one robot and one camera are used, but in an 
actual application, a plurality of robots or cameras 

so is used. 

A pattern plate 3 provided with a plurality of 
dot patterns is connected to an arm 2 of a robot 1. 
The details of the pattern plate 3 will be given later. 
The robot 1 is controlled by a robot controller 10. 

55 The robot controller 10 determines a coordinate 
position of the end of the arm on the robot's base 
coordinates, i.e., a coordinate position of TCP (Tool 
Center Point) as trw present position, and accord- 
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ingly. holds in a memory 11 the calibration pattern 
data on the robot coordinates which indicates the 
position of each dot pattern of the pattern plate 3 
based on the TCP of the arm 2 and the installation 
dimension of the pattern plate 3. The calibration 
pattern data is taken as a CPDr 11a. The robot 
controller 10 carries out a control such that the 
pattern plate 3 is not perpendicular to. but is at a 
certain angle with respect to, the optical axis of a 
camera 4. 

The camera 4 is connected to a visual sensor 
controller 20 which photographs the pattern plate 3 
by using the camera 4, to thereby calibrate the 
camera 4. 

The configuration of the visual sensor controller 
20 is centered around a processor (CPU) 21. Con- 
trol software 22a for implementing the calibration is 
stored in a ROM 22 to control the calibrating op- 
eration, and a RAM 23 stores calibration data (CD) 
23a, to be discussed later, and the calibration pat- 
tern data (CPDr) 11a received from the robot con- 
troller 10. The coordinate position data on each dot 
of the pattern plate and dot pattern data (DPD) 24a 
are stored in a RAM 24. 

The processor 21 picks up an image of the dot 
pattern on the pattern plate 3 through the camera 
4, in accordance with the control software 22a, and 
the image data is temporarily stored in a RAM 25 
via a camera interface 28. This image data is the 
video data of each dot pattern on the image sur- 
face of the camera 4. 

An image processing processor 26 obtains a 
calibration pattern data (CPDc) 25a of the camera 4 
from the position data and the already stored dot 
pattern data (DPD) 24a. and stores some in the 
RAM 25. 

The calibration pattern data (CPDr) 11a in the 
robot controller 10 is read from the interface 37 
through a communication ! : ne 13 and stored in the . 
RAM 23. 

Then, the calibration pattern data (CPDr) 11a 
on the robot cooidinates is compared with the 
calibutfon pattern data (CPDc) 25a on the camera 
coordinates, to thereby calculate the position and 
orientation of trv..- camera coordinate system with 
respect to the robot coordinate system, and thus 
perform the calibrator. The result is stored in the 
RAM 23 as the calibration data (CD) 23a. 

The calibration data (CD) 23a is used for as- 
sembling, palletizing and the like, and this data 
makes it possible to accurately determine the posi- 
tion and orientation of a workpiece in the robot 
coordinate system, through the camera 4 and the 
visv..:! sensor controller. 

FIG. 2 is the detailed view of the dot patt*- s 
on the pattern plate, wherein dot patterns 3a, 3b, 
3c and the like are arranged in the shape of a 
square on the pattern plate 3. Theoretically, six dot 



patterns should suffice, but 25 dot patterns are 
provided to obtain a more accurate calibration pat- 
tern data. The dot pattern 3a, in particular, is made 
larger than the other dot patterns, to serve as an 
5 origin. 

FIG. 3 illustrates an example where two cam- 
eras are used to pick up an image of one pattern 
plate. More specifically, by picking up an image of 
the dot patterns on the pattern plate 3 by two 
io cameras 5 and 6. the calibration of the respective 
camera coordinate systems can be carried out 
independently. 

FIG. 4 illustrates an example where, in a robot 
system consisting of four, robots and four cameras, 
;5 all four cameras are calibrated with respect to the 
robot coordinate system common to all the robots. 
An image of a pattern plate 3a of a robot 31 is 
picked up by a camera 41, an image of a pattern 
plate 3b of a robot 32 is picked up by a camera 42, 
20 an image of a pattern plate 3c of a robot 33 is 
picked up by a camera 43. and an image of a 
pattern plate 3d of a robot 34 is picked up by a 
camera 44. The picked-up image data of each 
camera is input to a visual sensor controller, not 
25 shown in this drawing, and the calibration data for 
the respective cameras is calculated. The details of 
this process are the same as in the case of FIG. 1. 
As may be also seen from this example, calibration 
data can be obtained for each camera, and there- 
30 fore, even when the position or the like of a camera 
is changed, only the calibration data of that camera 
need be obtained again. 

In the above explanation, an example wherein 
one robot and two cameras are used and another 
35 example wherein four robots and four cameras are 
used are described, but it is of course understood 
that those numbers may be changed as necessary 
Also, only one type of. pattern plate is required, 
and as need not be large, it can be manufactured 
40 easily. Further, there is no need to perform a 
jogging or the likri: to obtain calibration data. 

Although camera? are used as visual sensors 
in the abov3 description, other equipment such as 
las?- length measuring instruments may bo used to 
45 read patterns on a pattern plate, to thus obtain 
the ealib'ut on pattern data. 

As describee! above, according to the present 
invention, calibration pattern data in a robot system 
is sent to a visual sensor controller, which com- 
so pares that calibration pattern data with the calibra- 
tion pattern data on the visual sensor coordinates 
obtained by a visual sensor from the picked-up 
image data of a camera, to thereby obtain calibra- 
tion data, and thus the calibration data is easily 
55 obtained. 

Further, if the visual sensor picks up an image 
of the pattern plate, this will suffice and no special 
operation is required. Furthermore, a plurality of 
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visual sensors can be calibrated independently, 
and in addition, there is no need to use a special 
jig. thus eliminating the need for a jig installation 
space. 

5 

Claims 

1. A visual sensor calibration method of calibrat- 
ing of a visual sensor in a robot system com- 
prising; 70 

a pattern plate for obtaining a calibration 
provided on an arm of a robot, 

a step wherein first calibration pattern data 
of said pattern plate on a robot's base coordi- 
nates is sent from a robot controller to a visual /5 
sensor controller, 

a step wherein said visual sensor controller 
obtains second calibration pattern data from 
said pattern plate, and 

a step wherein calibration data is obtained 20 
from said first calibration pattern data and said 
second calibration pattern data, to thereby cali- 
brate visual sensors. 

2. The visual sensor calibration method according 25 
to claim 1, wherein said second calibration 
pattern data is obtained from an image picked 

up by a camera. 

3. The visual sensor calibration method according 30 
to claim 1, wherein said second calibration 
pattern data is obtained by a laser length mea- 
suring instrument. 

4. The visual sensor calibration method according 35 
to claim 1, wherein, from one robot's pattern 
plate, the second calibration pattern data is 
obtained by a plurality of cameras for each of 
said cameras, to thereby calibrate each cam- 
era. 40 

5. Tnc visual sensor calibration method according 
to claim 1. wherein said second calibration 
pattern Jala for a plurality of cameras is ob- 
tain^.! from pattern platea of a plurality of 45 
robots, to thereby Cciuy out a calibration. 

6. The visual sensor calibration method according 
to claim 1, wherein a pattern plate consisting 

of one dot pattern as an origin and a plurality so 
of dot patterns arranged in a square shape is 
used. 

7. The visual sensor calibration method according 

to claim 1, wherein said calibration data is 55 
obtained as matrix data. 
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FIG. 4 
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